2-D mesh-based synthetic transfiguration of an object with occlusion

This paper addresses 2-D mesh-based object tracking and mesh-based object mosaic construction for synthetic transguration of deformable video objects with deformable boundaries in the presence of another occluding object and/or self-occlusion. In particular, we update the 2-D triangular mesh model of a video object incrementally to account for the newly uncovered parts of the object as they are detected during the tracking process. Then, the minimum number of reference views (still images of a replacement object) needed to perform the synthetic trans guration (object replacement and animation) is determined (depending on the complexity of the motion of the object-to-be-replaced), and the trans guration of the replacement object is accomplished by 2-D mesh-based texture mapping in between these reference views. The proposed method is demonstrated by replacing an orange juice bottle by a cranbery juice bottle in a real video clip.